Notes on Estimation Theory

Stochastic (or random) variables are (mathematical) functions defined on the outcome of a random experiment. The experiments considered belong to a given class of experiments under consideration (e.g. dice throwing). The variable takes a real or complex value for each outcome, often called an event. For example, if you are throwing a dice and you win two points every time the outcome is even, while loosing a point when the outcome is odd, then the corresponding stochastic variable takes the value ’2’ for an experiment with even outcome and ’-1’ otherwise. Suppose that every outcome of the dice throwing is equally likely, then we assign to these outcomes a probability p of 1/6 (so that the overall probability of any event happening is 1), and the average for our even/odd stochastic variable becomes -1/6 + 2/6 1/6 + 2/6 -1/6 + 2/6 = 1/2. You would expect that when you do many such experiments, the overall outcome will approximate 1/2. We shall make this fact, which is known as the law of large numbers a little more precise soon.